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ABSTRACT 


This  brief  note  points  out  that  the  method  of  quadratic  interpolation, 
which  has  been  recommended  in  the  literature  for  minimizing  a function  of  one 
variable,  can  be  very  undependable.  In  particular,  unless  the  function  being 
minimized  is  itself  quadratic,  the  method  may  break  down  no  matter  how  close 
to  the  minimizer  one  starts. 
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SIGNIFICANCE  AND  EXPLANATION 


In  many  practical  applications  we  have  to  find  numerically  the  minimum 
or  maximum  value  of  a function  of  one  variable:  an  example  might  be  a 
statistical  estimation  problem  in  which  we  try  to  find  the  value  of  a parameter 
which  maximizes  the  likelihood  function. 

A simple  method  which  has  been  recommended  in  the  literature  is  to  find 
the  function  values  at  three  distinct  points,  then  fit  a quadratic  function 
(a  parabola)  to  these.  The  maximum  or  minimum  value  of  this  quadratic  function 
is  then  found,  together  with  the  point  at  which  it  is  attained,  and  this  point 
is  used  together  with  two  of  the  original  three  points  to  repeat  the  process. 

This  note  points  out  that  unless  the  function  being  minimized  is  itself 
quadratic,  the  above  algorithm  may  break  down  (the  new  point  may  be  one  of  the 
original  three  even  though  none  of  these  minimizes  the  function) . Even  if  the 
method  does  not  break  down,  it  may  provide  inaccurate  answers  because  of  the 
effects  of  roundoff  error.  Thus,  it  could  be  unwise  to  use  this  method. 


The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive 
summary  lies  with  MRC,  and  not  with  the  author  of  this  report. 
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Stephen  M.  Robinson 

Let  f be  a function  from  ® into  itself.  In  this  note  we  study  some  properties  of 
the  following  algorithm  for  finding  a minimum  of  f if  one  exists: 

QUADRATIC  INTERPOLATION  ALGORITHM:  Given  three  distinct  points  *0  ’*2 ' 

the  values  of  f at  these  points,  find  the  unique  polynomial  P of  degree  no  greater 
than  2 which  agrees  with  f at  Xj^  and  x^.  If  P has  a minimum,  let  x^  be 

a minimizer  of  P;  replace  one  of  x^,  Xj^,  x^  by  x^  to  obtain  three  new  points: 
stop  or  repeat  the  process. 

The  replacement  may  be  done  according  to  any  one  of  several  rules:  discard  the  "oldest" 
point,  discard  the  point  with  the  greatest  function  value,  etc.  Also,  various  stopping  rules 
may  be  applied  to  halt  the  algorithm  at  an  appropriate  step.  For  example,  see  the  discussion 
in  [1,  pp.  605-606),  in  which  the  rule  used  would  specify,  for  our  formulation,  that  the 
three  retained  points  z^  < z^^  < z^  satisfy  f(Zj^)  < min{f(zQ),  f[z^)).  It  is  stated  there 
that  the  use  of  this  rule  will  produce  a sequence  of  points  infinitely  many  of  which  are 
within  any  given  tolerance  of  a minimizer  of  f (the  authors  of  II)  phrase  their  discussion 
in  terms  of  meiximization,  but  this  has  no  effect  on  the  development)  . As  the  function  illus- 
trated in  their  Fig.  13.21  [1,  p.  605)  is  unimodal,  one  might  suppose  that  they  mean  for 
their  statement  to  be  interpreted  in  terms  of  such  a function.  No  discussion  is  given  in 
(1)  of  the  rate  at  which  the  asserted  convergence  occurs,  but  this  question  is  examined  in 
[2,  §2.3),  where  an  argument  is  made  with  a view  to  establishing  such  a rate. 

In  what  follows,  we  shall  show  that  unless  the  function  f is  itself  quadratic, 
this  method  may  brealc  down,  no  matter  how  close  to  a minimizer  of  f the  initial  points  are 
chosen:  that  is,  the  new  point  obtained  from  the  algorithm  may  be  one  of  the  current  three 
points  even  though  none  of  these  is  the  minimizer.  Next  we  discuss  briefly  the  numerical 
problems  to  which  the  method  is  subject  even  when  it  does  not  brea)<  down.  It  will  be 
apparent  from  these  considerations  that  the  quadratic  interpolation  method  is  unlikely  to  be 
the  algorithm  of  choice  in  most  applications. 

Sponsored  in  part  by  the  United  States  Army  under  Contract  No.  DAAG29-75-C-0024  and  in  part 
by  the  National  Science  Foundation  under  Grant  No.  MCS74-20584  A02. 


In  the  following  proposition,  we  suppose  that  we  are  dealing  with  a unimodal  function 


f:  i.e.,  one  for  which  there  is  a point  € F with  the  property  that  for  , x^  in  the 
domain  of  f,  if  *1^  ^ *2  - f(x^)  > f(X2),  and  if  ^ * ^2 

f(Xj^)  < f(x2)-  Evidently  x,  is  then  the  unique  minimizer  of  f. 

PROPOSITION:  Let  x^  « F and  let  E > 0.  Suppose  f is  a continuous,  unimodal 

function  from  the  interval  I :=  [x^  - E , x^  + e]  ^ F with  minimizer  x^.  Unless  f 

agrees  with  some  quadratic  function  on  I,  there  exist  points  ’'q  *1  * *^2  the  inter- 

val I,  with  x^  ^ X, , such  that  the  quadratic  function  Q interpolating  f at  these 
three  points  has  Xj^  as  its  unique  minimizer. 

Thus,  unless  f is  actually  quadratic  near  its  minimizer  x,,  the  algorithm  may  break 

down. 

PROOF:  We  distinguish  two  cases.  For  the  first  case,  suppose  that  there  is  some 
n £ (0,e]  with  f(x*  - h)  ^ f(x,  + n) . For  convenience  assume  that  f (x^,  - n)  < + h)  < 

and  let  x^:  = x^  - n.  As  f is  continuous  emd  strictly  increasing  on  (x^,  x^  + n) , there 

is  a point  x.  £ (x.,  x*  + n)  with  f(x,)  - (x  lies  in  the  open  interval  because 

f(x^)  < f(XQ)  < f(x*  + n)).  Now  let  Xj^:  = + *2''  evidently  x^  < Xj^  < x*  < x^,  so 

f(x^)  < f (Xq)  = f(x2)  by  unimodality.  It  is  easy  to  verify  that  the  unique  quadratic  func- 

tion 0 agreeing  with  f at  x^,  Xj^  and  Xj  attains  its  minimum  only  at  Xj^.  The  argu- 
ment for  the  case  ^ - n)  > f(x^  + n)  is  similar,  and  this  completes  the  proof  of  the 
first  case. 

In  the  second  case,  we  know  that  f (x^,  - h)  * h)  for  each  n e (0,e)  but  that 

f does  not  agree  with  any  quadratic  on  I.  For  brevity,  write  x^  :=  x^  - E and 

X :=  X + E.  Of  course,  f (x  ) = f(x  ) > f(x^).  Now  consider  the  family  of  quadratic 
functions  in  x defined  by 

C(y;x)  :-  ' 

-2- 


wtu-r('  V is  a <)iVL'n  point  in  (x  , x ).  The  function  0(v;-)  intetixDlates  f at  x and 

I.  K p 

at  y,  and  it  has  a unique  minimizer  at  y. 

Since  f does  not  agree  with  any  quadratic  on  lx  , x ] > it  certainly  does  not  agree 

L R 

with  Q(x^;  •)  there,  and  so  there  is  some  point  z e [x^^,  x^^J  with  f(z)  # Q(x^,i  z) . A 
quick  computation  shows  that  Q(x,;  x ) « f(x  ),  so  actually  z e (x  , x ),  and  by  symmetry 
we  can  suppose  that  z e (x,,  x^^)  . Assume  first  that  f(z)  < Qlx^;  z)  . Evidently  the  func- 
tion of  y given  by  Q(y;z)  is  continuous  for  y £ (x^^,  , so  we  can  find  Xj^  £ (x„,  z) 

with  Q(x  : z)  > f(z).  We  have  f (x  ) < f (x  ) by  unimodality  and  f(x  ) = f (x  ) by  symmetry, 
^ i-  L R L 

SO 


Q{x 


1' 


V - 


If(x^)  - f(Xj^)]  1 


< 0 . 


Thus  the  continuous  function  5(Xj^;  •)  - f(')  is  positive  at  z but  negative  at  Xj^,  so 
there  is  some  x^  £ {z,  x^^)  with  QlXj^;  x^)  = flx^).  Thus,  with  x^  :=  x^  we  find  that 
Xq  ^ x^  ^ ^2'  Q(Xj^;  •)  interpolates  f at  these  three  points  and  has  its  unique 

minimizer  at  x^^ , but  that  x^^  ^ 

If  f(z)  > Q(x^:  z)  then  we  choose  Xj^  e (x^,  x^,)  with  Q(Xj^;  z)  < f(z);  we  then 
find  that  QlXj^;  x^^)  > f(Xj^)  so  that  a similar  argument  applies.  This  completes  the  proof. 

We  have  shown  that  the  quadratic  interpolation  method  may  break  down;  even  if  it  does 

not  do  so,  numerical  difficulties  are  likely  to  prevent  one  from  finding  a very  accurate 

estimate  of  the  minimizer.  These  difficulties  are  well  known  to  any  experienced  numerical 

analyst.  They  are  shared  by  all  methods  using  only  function  values,  and  are  illustrated  by 

2 

the  problem  of  minimizing  1 + x ; it  is  easy  to  see  that  if  one  works  to  2N  slgnificeuit 

-N  -N 

figures,  the  computed  function  value  for  any  point  in  the  interval  1-2  *10  , 2 * 10  J 
will  be  1.  Thus  one  cannot  expect  to  be  able  to  determine  the  minimizer  to  within  more  than 
£dx3ut  half  the  number  of  significant  digits  carried  by  the  machine.  This  loss  of  significance 
does  not  occur  with  methods  which  attempt  to  find  a zero  of  the  first  derivative  (e.g., 
Newton's  method  or  the  secant  method),  since  near  a nondegenerate  minimizer  the  absolute 


-3- 


value  of  the  first  derivative  will  be  roughly  proportional  to  the  distance  to  the  minimizer 
instead  of  to  the  square  of  that  distance.  Thus,  when  derivatives  are  available  it  is 
generally  wise  to  make  use  of  them.  Even  when  only  function  values  can  be  computed,  the  use 
of  a reliable  search  method  (e.g.,  golden  section  search)  seems  likely  to  be  safer  than 
quadratic  interpolation  because  search  algorithms  axe  not  subject  to  the  breakdown  property 
discussed  above. 


» 
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